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141/00863 

IMAGE COMPRFSSION 
FIELD OF THE INVENTION 

The present invention relates to the field of optical processing and especially to 
compressing images using optical components. 

BACKGROUND OF THE INVENTION 
Often, the information bandwidth to be transmitted is greater than the available 
bandwidth. Therefore, information is often compressed before it is transmitted (or stored), to 
reduce the required bandwidth. For example, the HDTV standard was defined, at its inception, 
to include compression. Many types of signals are compressed, including still images, video 
and telephone conversations. The reigning compression standards are JPEG for still images 
and MPEG (I, II, III or IV) for video. In actuality, these standards are standards for the 
compressed signals. There is no particular requirements, in the standard, on the method for 
converting the uncompressed signals into compressed signals. 

Compression and in some cases decompression are often very demanding and typically 
require dedicated hardware. Both JPEG and MPEG are transform-based methods, in which the 
uncompressed data is transformed into a transform space, where the data is represented by a 
set of coefficients. It is usually desirable that the coefficients have less autocorrelation than the 
image data or even no autocorrelation at all. Although the DCT transform does not completely 
decorrelate the coefficients, the correlation between them is significantly reduced. In other 
compression methods, other transform spaces are used. In transform space, some of the 
coefficients have a greater visual and/or other effect on the image, than other coefficients. To 
obtain compression, the coefficients are quantized, with fewer bits being allocated to those 
coefficients which have a lesser effect. Typically, a coefficient is quantized by dividing it by a 
weight and then rounding or truncating the result. 

Optical and electro-optical processors have been used in the art, to a small extent, for 
computationally demanding applications. However, with the advent of very fast electronic 
computer components and parallel processors, their acceptance has been limited. 

Performing some types of linear transforms, for example Fourier transforms, 
continuous cosine transforms and Walash transforms, using optical components is well known, 
for example, as described in "Cosinusoidal Transforms in White Light", by N. George and S. 
Wang, in Applied Optics . Vol. 23, No. 6, March 15, 1984, in "Hartley Transforms for Hybrid 
Pattern Matching", by Nomura, K. Itoh and Y. Ichioka, in Applied Optics, Vol. 29, No. 29, 
October, 10, 1990, in "Lens Design for a White-Light Cosine-Transform Achromat", by K. B. 
Fan and S. Wang, in AppUed Opfics, Vol. 34, No. 1, January 1. 1995 and in "Optical 



1 



10 



15 



20 



25 



141/00863 

Computing", by D. Feitelsfl^n a chapter titled "Optical Image aAgnal Processing", pp. 
102-104 (which pages describe general discrete linear transforms using a lenslet array), and pp. 
1 17-129 (which describe matrix multiplication), MIT Press 1988, the disclosures of which are 

incorporated herein by reference. 

SUiMMARY OF THE INVENTION 
An object of some preferred embodiments of the invention is efficient and/or fast 
generation of compressed still and moving images, especially transform based compression 
methods. In a preferred embodiment of the invention, this object is achieved by using optical 
components which perform specific steps of a compression method. Some compression 
methods utilize models, e.g., of visual perception to provide lossy compression. In a preferred 
embodiment of the invention, the optical components used are selected to match physical 
processes and/or models which are mimicked by the compression method. Alternatively or 
additionally, the optical components are selected to perform computation-intensive steps, for 
example transforms, which can be performed effectively using optical components. 

A compression system in accordance with a preferred embodiment of the invention 
preferably comprises five stages, an input which receives the data to be compressed, an 
optional pre-processing stage which converts the representation of the data into a presentation 
more suitable for processing, a processing stage which performs the compression, an optional 
post processing stage which converts the representation of the compressed data into one 
suitable for output and an output stage which outputs the data. In a preferred embodiment of 
the invention, some or all of the stages are optical. In some embodiments, one or more 
electronic or hybrid electronic and optical stages may be used, for example for pre-processing 
the data. Additionally, in some embodiments, only some of the processing is performed 
optically, with the balance of the processing preferably being performed electronically. 

An aspect of some preferred embodiments of the invention relates to optical block 
transforms, especially of image data. In a preferred embodiment of the invention, an optical 
component is used to transform image data in blocks, with each block being transformed 
separately. In a preferred embodiment of the invention, the transform used is a DCT (Discrete 
Cosine Transform) transform, preferably a JPEG-DCT, which is the DCT transform variant 
used for JPEG. Alternatively or additionally, the transform is a wavelet transform, such as the 
S transform, S+P transform family, other integer to integer "wavelet-like" transforms, or 
general known wavelet transform (Daubechies etc.) useful for wavelet compression. 

Preferably, the block size is 8x8, which is a standard block size for many applications. 
Alternatively, different block sizes may be used, for example 16x16 or 64x64, possibly with 
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different block sizes and/or block aspect ratios for different parts of the image. For wavelet 
transforms, larger blocks are preferably used. 

An aspect of some preferred embodiments of the invention relates to performing a 
DCT (Discrete Cosine Transform) using optical processing, preferably a JPEG-DCT. In a 
5 preferred embodiment of the invention, a single optical element is used to transform data from 
an image domain to a DCT domain. A related aspect is an optical element which performs 
discrete wavelet and "integer to integer" wavelet transforms (such as the S and S+P 
transforms), for example using a combination of diffraction gratings neutral density filters (for 
weighting sums and differences). 
10 An aspect of some preferred embodiments of the invention relates to a block-DCT- 

transforming lens, preferably a JPEG-DCT performing lens. In a preferred embodiment of the 
invention, such a lens comprises a two dimensional mafrix of groups of optical elements , 
each such group performing a DCT on a single block. Preferably, such a group comprises a 
lenslet array which performs the DCT directly. Alternatively, the matrix comprises a matrix of 
15 optical elements , with each optical element performing a DCT transform for a single block. 
AUematively to performing a DCT transform, a correspondence between JPEG DCT and DFT 
(Discrete Fourier Transform) may be utilized, so that a Fourier-transforming lens (or optical 
element or lenslet array) is used. Preferably, optical or electrical components are provided to 
modify the data and/or the transformed data so that the Fourier lens generate a DCT transform, 
20 at least for real image data. Alternatively to block-DCT lens, a lens for performing other types 
of block transforms, such as a block- Wavelet-transform, can be provided. 

An aspect of some preferred embodiments of the invention relates to performing 
optical motion estimation. In a preferred embodiment of the invention, the motion estimation 
is performed on block-DCT-fransformed data, by comparing DCT coefficients of neighboring 
25 blocks. Preferably the same hardware is used to perform DCT for motion estimation and for 
image compression. Alternatively or additionally to motion estimation, motion compensation 
may also be performed by correcting DCT coefficients of transformed data. 

An aspect of some preferred embodiments of the invention relates to data compression 
using optical components. In various preferred embodiments of the invention, individual steps 
30 of image compression methods are performed using optical components. In some preferred 
embodiments of the invention, multiple sequential steps are implemented using optical 
components, possibly without conversion back to electrical signals in-between steps. 

In a preferred embodiment of the invention, the data compressed is image data. 
Preferably, compression method is a fransform based method, especially a DCT based method, 
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such as JPEG or MPEG. /Amatively or addiiionally, other type o^fta compression which 
require processing (not spatial zooming) may be used, for example, entropy encoding. In a 
preferred embodiment of the invention, at least the DCT and/or motion estimation steps used 
for the above compression methods are performed optically. Alternatively, the compression 
method is a wavelet based compression method. 

Alternatively or additionally to compression, data decompression may be effected 
using optical processing, for example to perform an inverse DCT. 

.An aspect of some preferred embodiments of the invention relates to direct acquisition 
of images which are compressed, partially compressed or pre-processed for rapid compression. 
In a preferred embodiment of the invention, a camera uses a DCT-transforming lens, which 
receives light from an imaged object and projects a transform of the light onto an optical 
detector, such as a CCD, for data acquisition. Alternatively, other types of optical detectors, 
such as a CMOS detector may be used. Preferably, but not necessarily, other optical elements 
are provided between the DCT lens and the CCD to perform further optical processing and'or 
image compression on the data. Alternatively or additionally, optical and/or electro-optical 
elements are provided between the object and the DCT lens to perform pre-processing on the 
optical data, for example to change its data representation scheme, or to better process 
polychromatic light. In a preferred embodiment of the invention, the DCT lens accepts 
polychromatic light. Alternatively, color information is separated out of the light received 
from the imaged object and the DCT lens accepts monochromatic light. In a preferred 
embodiment of the invention, the optical processing is use to perform or aid in performing 
JPEG or MPEG compression. Alternatively or additionally, other compression protocols are 
performed. Alternatively to DCT lens, other block-transform lens may be provided, for 
example for an S-transform. 

An aspect of some preferred embodiments of the invention relates to using a 
continuous Fourier-transform optical system, for example a Fourier lens, for performing a 
discrete transform. In a preferred embodiment of the invention, data to the Fourier lens is 
matched to a data receptor at the other side of the Fourier lens to allow a discrete transform to 
be performed. 

An aspect of some preferred embodiments of the invention relates to determining a 
matching between an SLM and a CCD to allow the use of Fourier optics for performing 
transforms, such as linear transforms. In a preferred embodiment of the invention, the size of 
elements in the CCD and the SLM is matched. Alternatively or additionally, one or more 
continuous spatially varying neutral density filter is used to match the CCD to the SLM. 
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There is thus provided in accordance with a preferred embodiments of the invention, a 
method of image compression, comprising: 
providing image-data encoding light; 

transforming said light from an image space to a transform space utilizing an optical 
component; and 

converting said transformed light into electrical signals, which electrical signals 
represent a compressed representation of said image data. Preferably, said converting 
comprises converting after quantization. 

There is also provided in accordance with a preferred embodiment of the invention, a 
method of image de-compression, comprising: 

providing compressed-image-data encoding light; 

transforming said light from a transfonm space to an image space utilizing an optical 
component; and 

converting said transformed light into electrical signals, which electrical signals 

represent said image data. 

In a preferred embodiment of the invention, said transforming comprises transforming 
using one or more optical elements which perform a block transform. Preferably, said one or 
more optical elements comprises a matrix of optical lens elements. Preferably, at least one of 
said optical lens elements comprises a lenslet array which performs a block transform for a 
single block. 

In a preferred embodiment of the invention, said one or more optical elements 
comprises an element which performs a vector by matrix multiplication. Alternatively, said 
one or more optical elements comprises an element which generates a matrix product. 
Preferably, said matrix product comprises a triple matrix product. 

In a preferred embodiment of the invention, said transforming utilizes at least one 
electrical component for transferring data between said at least one optical component. 

In a preferred embodiment of the invention, said image is compressed using a JPEG 
standard. Alternatively, said image is compressed using an MPEG standard. Alternatively, said 
image is compressed using a wavelet compression method. 

In a preferred embodiment of the invention, said image data represents an image 

sequence. 

There is also provided in accordance with a preferred embodiment of the invention, 
apparatus for direct acquisition of compressed image data, comprising: 

at least one optical element which receives light waves from an imaged object and 
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which transforms said imag^nto a transform space; and 

an optical detector which receives said transformed light and generates electronic 
signals responsive to said received light. Preferably, the apparatus comprises an optical 
quantizer which quantizes said transformed light prior to its reception by said detector. 
5 Alternatively, the apparatus comprises an electronic quantizer which quantizes said electromc 
signals to produce compressed data. Alternatively, said optical detector quantizes said light. 

Alternatively or additionally, said at least one optical element comprises a single 
optical element which performs said transforming. Alternatively or additionally, the apparatus 
comprises a storage unit for storing at least one set of said electronic signals, corresponding to 
,0 a still image. Ahematively or additionally, the apparatus comprises a storage unit for storing at 
least one set of said electronic signals, corresponding to an image sequence. 

In a preferred embodiment of the invention, said transform space comprises a DCT 
transform space. Alternatively, said transfonn space comprises a wavelet transform space. 

There is also provided in accordance with a preferred embodiment of the invention, a 
15 block-transform optical element, comprising: 

a pluralitv of optical groups each group comprising at least one optical sub-element, 
wherein each of said optical groups transforms a block of data. Preferably, said at least one 
sub-element comprises a lenslet array portion. Alternatively or additionally, said plurahty of 
optical groups are arranged as a matrix. Alternatively or additionally, said element performs a 
20 JPEG-DCT transform. Alternatively or additionally, said element performs an inverse DCT 
transform. Alternatively or additionally, said element performs a wavelet transform. 
Alternatively or additionally, said optical element performs said transfonn on data arranged as 

a spatially impulse image. 

In a preferred embodiment of the invention, said optical element performs said 
25 transform on data arranged as a continuous image. Alternatively or additionally, said transform 
comprises a linear transform. Alternatively or additionaUy, said transfonn compnses a non- 
separable transfonn. Alternatively or additionally, said optical groups overlap. Alternatively or 
additionally, said optical groups no not overlap. 

There is also provided in accordance with a prefened embodiment of the invention, a 
30 method of DCT-like transforming of data, comprising: 

providing optical waves which encode said data; and 

passing said waves through at least one optical element, which optical element 
modifies said optical waves into second optical waves carrying a transfonn of said data. 
Preferably, said transfonn comprises a block transfonn. Alternatively or additionally, said at 
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least one optical element is an atomic optical element. Alternatively or additionally, said at 
least one optical element comprises a plurality of optical elements. Alternatively or 
additionally, said at least one optical element comprises a discrete optical processing element. 
Alternatively, said at least one optical element comprises substantially only continuous optical 
processing element. 

In a preferred embodiment of the invention, said transform is a JPEG-DCT transform. 
Alternatively, said transform is an inverse DCT transform. 

There is also provided in accordance with a preferred embodiment of the invention, a 
method of optical motion estimation, comprising: 

providing a first set of Hght waves encoding data of a first image block; 

DCT transforming said first set of light waves using an optical component; 

providing a second set of light waves encoding data of a second image block; 

DCT transforming said second set of light waves using an optical component; and 

processing said first and said second DCT transforrhed data sets to determine an 
amount of relative motion of said two blocks. Preferably, both of said DCT transforms utilize a 
same optical component for said transforming. Alternatively or additionally, said processing 
comprises electronically processing said transformed data sets. Alternatively or additionally, 
the method comprises utilizing said determined motion for generating a compressed image 
stream. 

There is also provided in accordance with a preferred embodiment of the invention, a 
method of transforming data using a discrete transform, comprising: 
preprocessing said data to yield optically encoded data; 
Fourier transforming said data using a Fourier lens; and 

postprocessing said data to yield transformed data. Preferably, said preprocessing 
comprises optically preprocessing. Preferably, said preprocessing comprises matching said 
data to said postprocessing. Preferably, said postprocessing comprises acquiring said data and 
wherein said matching comprises matching to said acquisition. Preferably, said postprocessing 
comprises spatially modulating said light. 

In a preferred embodiment of the invention, said preprocessing comprises spatially 

modulating said light. 

In a preferred embodiment of the invention, said spatially modulating comprises 
masking said data using an array of pinholes. Altematively, said spatially modulating 
comprises continuously varying said amplitude of said data. 

In a preferred embodiment of the invention, said preprocessing comprises mirroring 



141/00S63 

said data. Alternatively o^^iiionally, said transform comprise^fcpEG-DCT transform. 
Alternatively or additionally, said transform comprises a wavelet transform. Alternatively or 
additionally, said transform comprises an inverse DCT transform. 

There is also provided in accordance with a preferred embodiment of the invention, 
apparatus for compressed image data manipulation, comprising: 

an input for receiving data; 

at least one optical component for transforming said data between an image space and a 
transform space; and 

an output for outputting said manipulated data. Preferably, said received data 
comprises uncompressed image data and comprising a processor for processing said data after 
said transform to yield compressed image data to be outputted. Preferably, said received data 
comprises compressed image data and comprising a processor for processing said data before 
said transform to yield transform data to be transformed by said optical component for said 
output. Preferably, said processor includes an optical D/A converter. Alternatively, said 
processor comprises an optical multiplier for de-quantizing data. 

In a preferred embodiment of the invention, said processor performs at least some of its 

processing on optic waves. 

In a preferred embodiment of the invention, said processor performs at least some of its 
processing on electronic signals. Altematively or additionally, the apparatus comprises an 
optical motion estimator. Preferably, said optical motion estimator utilizes said optical 
component for optical motion estimation. Altematively or additionally, said optical motion 
estimator includes at least one electronic component. 

In a preferred embodiment of the invention, the apparatus comprises an optical motion 
compensator. Preferably, said optical motion compensator utilizes said optical component for 
optical motion compensation. Altematively or additionally, said optical motion compensator 
includes at least one electronic component. Altematively or additionally, said data is 
compressed using a JPEG compression. Altematively, said data is compressed using a wavelet 
compression. 

In a preferred embodiment of the invention, said data is compressed using an MPEG 
compression. Altematively or additionally, said data comprises an image sequence. 

In a preferred embodiment of the invention, said data comprises is a single image. 
Altematively or additionally, said data is manipulated as blocks. Preferably, at least two of 
said blocks are manipulated in parallel. Altematively, at least two of said blocks are 
manipulated in series and comprising a buffer circuit for directing said blocks in series into a 
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single optical component. 

In a preferred embodiment of the invention, said optical component comprises a 

discrete data transforming optical element. Alternatively or additionally, said optical 

component comprises a holographic lens. Altematively or additionally, said optical component 
5 comprises an acousto-optic element. Altematively or additionally, said optical component 

comprises an optical analog to digital converter. Altematively or additionally, said optical 

component comprises an optical digital to analog converter. Altematively or additionally, said 

optical component comprises matrix product calculating element. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 The present invention will be more clearly understood from the following detailed 

description of the preferred embodiments of the invention and from the attached drawings, in 

which: 

Fig. 1 is a flowchart of a baseline method of JPEG-compUant compression; 
Fig. 2 is a schematic block diagram of an optical JPEG compression system, in 
15 accordance with a preferred embodiment of the invention; 

Fig. 3 A is a schematic block diagram of a matrix-multiplication based optical DCT 
component, in accordance with a preferred embodiment of the invention; 

Fig. 3B is a schematic block diagram of an optical matrix by vector multiplication 
component, in accordance with a preferred embodiment of the invention; 

Fig. 4A is a schematic diagram of a lens-matrix based 2D DCT component, in 
accordance with a preferred embodiment of the invention; 

Fig. 4B is a schematic diagram of an optical element for the lens matrix of Fig. 4A; 

Fig. 4C is a schematic diagram of a lenslet array for the lens matrix of Fig. 4 A; 

Fig. 4D is a schematic illustration of an optical system for performing a DCT transform 

25 using a Fourier lens; 

Fig. 5 is a schematic block diagram of an optical JPEG compression system, in 
accordance with another preferred embodiment of the invention; 

Fig. 6 is a schematic flowchart of a base-line method of MPEG-compUant 
compression; 

Fig. 7A is a schematic diagram of a direct-compression camera system, in accordance 
with a preferred embodiment of the invention; 

Fig. 7B is a schematic block diagram of a YUV-separated implementation of the 

embodiment of Fig. 7 A; and 

Fig. 8 is a schematic diagram of a lithographic implementation of an optical 
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compression system in accoXce with a preferred embodiment of tHW^'ention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
Fig. 1 is a flowchart of a base-line method 20 of JPEG-compliant compression. Image 
data is first transformed using the DCT (Discrete Cosine Transform) (22), to generate a set of 
coefficients. These coefficients are then quantized (24). The quantized coefficients are then 
unfolded from a 8x8 representation to a 64x1 representation ("Zig-Zag", 26). These quantized 
coefficients are encoded using a variable-length encoding scheme (28), zero-run length 
encoded and then Huffman encoded (30), to reduce entropy. A compressed data file is then 
generated by prefixing the encoded data with header information (32). Other, similar, methods 
of JPEG compression are also known. 

In accordance with a preferred embodiment of the invention, various of the above steps 
are performed using optical elements, rather than using electronic or software elements. In the 
above described JPEG compression method, the step that is. typically most computationally 
demanding, is the DCT step. Thus, in a preferred embodiment of the invention, the DCT step 

is performed optically. 

Fig. 2 is a schematic block diagram of an optical JPEG compression system 50, in 
accordance with a preferred embodiment of the invention. An electronic input 52 is preferably 
used to receive the data to be compressed. This element will generally depend on the 
application for which system 50 is used. For example, if system 50 is implemented on a PC 
card, electronic input 52 will generally comprises a PC-compatible bus connector. The 
acquired data is then converted into light, using a spatial light source 54. In a preferred 
embodiment of the invention, light source 54 comprises an SLM (Spatial Light Modulator) 
which modulates a beam of light that is reflected from it or that is transmitted through it. 
Altematively, source 54 may comprise an array of LEDs, laser diodes, VCELs (vertical cavity 
emission lasers) or other types of pixelated display devices such as CRT, field effect emitter 

arrays and plasma displays. 

The type of light emitted by source 54 is preferably selected to match an optical DCT 
unit 56. In some preferred embodiments of the invention, the light from source 54 is coherent 
(so a laser source is preferably used). In other embodiments, the optics do not require coherent 
light. In a preferred embodiment of the invention, the light is preferably monochromatic. 
Altematively, polychromatic light may be used. In some particular preferred embodiments of 
the invemion, multiple frequencies of monochromatic light are used, for example wherein the 
frequencies are used to encode attributes of the data, such as its numerical sign. In a preferred 
embodiment of the invention, the data is encoded using an analog encoding scheme, for 
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example phase or amplitude. Altematively, a digital encoding scheme is used. Possibly, as 
described below, the light may be A/D converted from analog-encoding light into digital- 
encoding light, for example after it is transformed. 

Optical DCT unit 56 transforms the light from an image space to a transform space. 
Preferably, the transformed hght is projected unto a spatial optical sensor 58, such as a CCD 
array. Details of various types' of DCT unit 56 and methods of construction thereof are 
provided below. 

Data is read out of CCD array 58 and then it is quantized, using a quantizer 60. In a 
preferred embodiment of the invention, the quantization may be performed by setting gain 
and/or offset characteristic of the CCD and/or individual elements thereof and/or controlling 
the readout of the CCD, for example to provide a reduced bit-per-pixel ratio. Alternatively, the 
data is quantized as it is read off the CQD or after it is read off the CCD. Alternatively or 
additionally, the data is quantized by optical means, such as a second SLM in front of the 
CCD. The unfolding of the data may be performed before the quantizing or after the 
quantizing. Then, the data is encoded using a variable length encoding unit 62, Huffinan- 
encoded using a Huffman encoding unit 64 and, finally, a header is attached to the compressed 
data so that it meets the JPEG standard. Alternatively or additionally, the data is encoded using 
arithmetic coding (preferably performed by an arithmetic coding unit - not shown). 

As will be described below, additional elements of the compression system may be 
replaced with optical units. In some embodiments of the invention, the different optical units 
will be interconnected with electrical circuitry, for example for control, data management or 
data conversion. Thus, even if two consecutive units are embodied using optical means, they 
may have an intervening step of optical/electrical conversion and then electrical/optical 
conversion. In other embodiments, the processed Hght will feed directly from one optical unit 
to the next. In a preferred embodiment of the invention, a system includes both optical and 
electronic components and the processing is divided between the components so they can act 
in parallel. In one example, some of the transforming may be performed optically and some 
electronically. Such dividing up of work, can better utiUze all the elements in a 
compression/decompression device, especially if some of the components are dual use, for 
example DSP components. 

Optical DCT unit 56 may be implemented in various ways. It should be noted that 
when compressing images, the DCT transform applied is in actuaUty a block-DCT transform, 
where each part of the image is separately transformed. 

Fig. 3A is a schematic block diagram of a matrix-multiplication based optical DCT 
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component 70, in accordance with a preferred embodiment ofW^ mvention. The DCT 
transform can be presented in matrix form as [DCT]=[C][T][C]. Matrix by matrix 
multiplication may be performed in many ways, including using multiple repetitions of vector 
by matrix multiplication, for example as described in "Introduction to Fourier Optics". 
Goodman, pp. 286, or using direct matrix by matrix multiplication, for example as described in 
Feitelson, pp. 126-127 (double or triple products), preferably using monochromatic coherent 
light, or as described in Feitelson, pp. 118, using lenslet arrays, which can accommodate white 
lisht; the disclosure of all of which is incorporated herein by reference, 

In a vector by matrix embodiment of component 70, a line data provider 72 provides 
individual lines or columns of an 8x8 block to a matrix multiplier 74. The DCT transform of a 
vector can be performed by multiplying a source vector V by a convolution matrix C, to obtain 
a transformed vector T. For each 8x8 block the lines (or the columns) are individually 
transformed and then the result is transformed along the individual columns (or lines). In a 
preferred embodiment of the invention, the data is row transformed using a first unit 74 and is 
then column transformed using a second multiplication unit 74'. Alternatively, a same unit is 
used for both the row and column transforms. Preferably, the transformed row data is 
accumulated using a store unit 76. If each one of the rows is transformed in sequence, the 
transformed row data may be accumulated using store 76 even if a separate unit 74' is used for 
column transforms. 

Fig. 3B is a schematic block diagram of an optical matrix-by-vector multiplication 
component 80, in accordance with a preferred embodiment of the invention. When performing 
a DCT transform, negative-valued results may be produced. Multiphcation component 80 
separately processes negative- and positive- valued results, to avoid miss-processing. 
Mathematically, the multiplication of a matrix C by a vector V is a linear operation, so that it 
can be separated in to negative and positive components, e.g.: C*V=Cp*Vp-HCn*Vn-Cp*Vn- 
Cn*Vp, where the "n" subscript indicates negative numbers and the "p" subscript indicates 
positive numbers. In the component of Fig. 3B, vector V is separated into positive and 
negative values, which are each separately multiplied by positive or negative valued 
component matrixes 82 and then summed using subtracters 84 and an adder 86. In a preferred 
embodiment of the invention, four matrix multiplication units 82 are provided. Alternatively, 
only two or even only one unit 82 is used, for example to sequentially process negative and 
positive numbers. In general, the source data is all positive, so that the vector Vn is empty. It is 
noted that the DCT of the original image data, which is positive, may be simpler to implement 
than the DCT of transformed data, which may be negative. 
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In a preferred embodiment of the invention, sign issues are solved using a bipolar 
number representation. In a bipolar representation, each number is designated by two 
components: s=[p,n], where s is a general signed number (not necessarily an integer), and [p,n] 
are its positive and negative components, s is retrieved by setting s=p-n. Therefore, the number 
-5 can be described by [0,5], [3,8], [10,15], [1,6] or other combinations of p and n, as long as 
p, n > 0, and p-n=s. 

The generalized bipolar representation can be adapted to matrix calculation, by 
representing each number by a 2x2 matrix, of the form 
[pn] 



[np]. 



For example: 
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"-3 


-5' 


3 -4 




2 2 




-5 





"1 


0 


0 


2" 




'1 


0 


0 


1 ■ 




'1 


4 


0 5 ■ 


0 


1 


2 


0 




0 


1 


1 


0 
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11 0_ 



This representation can be extended to triple product matrix multiplication. 

Alternatively to the method of Fig. 3B, negative numbers may be dealt with in other 
ways. In one example, negative and positive numbers are differentiated by their phase. A 
diffraction grating can be used to divert numbers with one phase value to a different part of a 
CCD target (or other optical element) than those numbers with a second phase values. In 
another example, negative numbers are encoded using a different frequency than positive 
numbers. The different frequencies can be separated using suitable gratings or other optical 
elements. Alternatively or additionally, a self-electro-optical device may use one frequency in 
order to modulate the other frequency. Alternatively or additionally, a frequency sensitive 
CCD may be used, for example an RGB CCD. Altematively or additionally, a CCD may be 
provided with binary phase or frequency detection, by providing a controllable polarizer or 
spectral filter before the CCD and timing its operation to the acquisition of positive or negative 
numbers. 

Altematively, negative numbers may be managed by biasing them to be positive, for 
example, by forcing the results of a DCT to be in the range [0...2] instead of [-1...1] 
(normalized values). In practice, if the maximum DC amplitude is A, the DCT results are 
shifted by +A, from the range [-A. . .A] to the range [0...2A]. In the example (described below) 
where a DCT is performed by mirroring the 8x8 datablock into a 16x16 datablock, a strong 
spatial delta pulse is provided in the middle of each 16x16 datablock, for example by 
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controlling the SLM. The^ects of this pulse (the bias) are ^fcrably removed using 
electronic processing after the data is transformed. 

Once the data is multiplied, further processing, such as sign extraction or as described 
below, can be achieved, for example, by performing optical A/D, and then binary operations or 

5 by using electronic components. 

In the above description, a plurality of matrix-by-matrix or vector-by matrix operations 
are performed. The number of actually provided multiplication units depends on the 
implementation and especially on the level of parallelism of the implementation. For example, 
in one implementation, all the 8x8 blocks are processed in parallel. However, within each 

10 block, the multiplications may be performed in parallel (using a plurality of units if required) 
or in sequence (reusing a single unit for two operations, for example for row and for column 
DCT). Alternatively or additionally, two or more of the blocks may be processed in series, for 
example the two blocks sharing a single 8x8 multiplier. Such sequential processing generally 
requires electronic components, such as store 74, to read and store intermediate results and 

15 possibly also for summing up the individual results. 

In a preferred embodiment of the invention, each matrix multiplication unit comprises 
a series of {SLM, lens, CCD} sub-systems which unit accepts electronic data at one end, 
converts it into optical signals, transforms the data using lens and then converts the 
transfonned data into electronic signals again. Alternatively, a single SLM and/or a single 

20 CCD may be shared among several multipliers. 

Fig. 4A is a schematic diagram of a lens-matrix based 2D DCT component 90, in 
accordance with a preferred embodiment of the invention. Light from an image of an object 92 
impinges on a lens-matrix 94. Array 94 preferably comprises a plurality of lens-elements 95. 
each of which performs a DCT on one 8x8 block of image 92. The result of the DCT is 

25 recorded by a CCD 96. 

Fig. 4B is a schematic diagram of a single optical element 98 suitable for the lens 
matrix of Fig. 4A, for performing a DCT. Optical element 95 is designed so that light emitted 
by different portions of the lens corresponds to different coefficients of the DCT transform of 
the impinging light. Thus, Ught corresponding to a first DCT coefficient is detected by CCD 

30 96 at a point A. Light conresponding to a second DCT coefficient is detected at a point B. 
Typically, at least some of the light emitted by lens 95 does not correspond to a DCT 
coefficient, due to design considerations. Such light may be detected, for instance at a point C. 
The readout of CCD 96 is preferably configured to account for the correspondence between 
DCT coefficients and spatial locations on CCD 96. In a preferred embodiment of the 
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invention, a plurality of or even all of optical elements 98 are combined into a single 
composite optical element. Alternatively, a single optical element 98 may be implemented as a 
sequence of individual optical elements. 

Fig. 4C is a schematic diagram of a lenslet array 100 for the lens matrix of Fig. 4A, for 
performing a DCT. In a lenslet array, each individual lenslet preferably generates one DCT 
coefficient from the impinging light. In one exemplary embodiment, light from a 8x8 block of 
the image is received by 64 lenslets, preferably arranged in a 8x8 array. After each lenslet is a 
mask having opaque and transmissive portions and a CCD element is positioned opposite the 
mask to receive light which passes through the mask. In an exemplary embodiment, each 
lenslet creates an image of the image to be transformed. Each DCT coefficient d(k,l) is defined 
as: 

M N 

d(k,l) = 'ZJLf(U)Kk,l;iJ) (1) 
1 1 

where f is the input and h is the convolution definition. The opaque and transmissive portions 
of each of the (k,l) masks are defined to represent the values of h, in which the 
transmissiveness of mask elements for a lenslet (k,l) are defined to match the relative 
contribution of those image pixels (i,j) which take part in determining the (k,l) coefficient. The 
CCD element sums the light which passes through the mask, determining the DCT coefficient. 

The Formula for a DCT transform and for an inverse DCT transform of an 8x8 block of 
image data f(x,y) and an 8x8 block of transform data F(u,v), are, respectively: 

1 J, J, (2.x + 1)m^ (ly + l)v/r 
F(«,v) = -C(u)C(v) X Z fi^>y)<^os cos 

x=Oy=0 

fi^,y) = J Z Z C(«)C(v)/r(„,v)cos^-j^ cos 

'*u=Ov=0 

where C(u) and C(v) are 1/V2 for u,v=0 and 1 otherwise. 

In a preferred embodiment of the invention, a single large lenslet array is used to 
implement a matrix of individual lenslet arrays. In a preferred embodiment of the invention, 
the light from object 92 is focused onto lens matrix 94, so that all of the object is viewed by 
each one of the lens elements 95. Alternatively, only a block area of the light impinges on each 
lens element, for example by providing multiple side-by-side lens, each one viewing only a 
portion of object 92. Alternatively, where an SLM is used, the light which passes through the 
SLM can be formed of blocks of non-parallel light, so that block portions of the modulated 
light impinge each on a different lens element 95. Alternatively, especially where individual 

15 



141/00863 

optical elements are used, tWight from object 92 can be parallel so that each optical 

element receives parallel light from a single block area. In some cases, a lens element may 
receive light from more than one block area, for example for processing which is beyond the 
extent of a single block or to provide an overlap between blocks, for example to solve 
calibration problems. 

Fig. 4D is a schematic illustration of an optical system for performing a DCT transform 
using a Fourier lens. Although in one preferred embodiment of the invention the lens- element 
directly performs a DCT, in an alternative preferred embodiment of the invention, a 
correspondence betv^'een Fourier transfonn and DCT is utilized to perform DCT (or other 
transforms) using a Fourier transform lens. Mathematically, a Fourier transform of real and 
symmetrical data results in only real (and symmetric) cosine coefficients. The image data to be 
compressed is typically real. It can be made symmetric by mirroring in the SLM. In order to 
achieve a discrete transform, the data is preferably provided as an impulse image, with each 
image portion being a spatial delta function , each of which pulses is transformed using a 
Fourier transform lens. This type of data provision can be achieved using an SLM with a 
pinhole filter. In multi-wavelength based embodiments, different pinholes may be designated 
for different wavelengths. Preferably both the SLM and the CCD are spatially matched 
according to the following formula: Ax = 0.5Ax , which defines the distances between the delta 

functions (pinholes) in the SLM and " 2- Ax N , which defines the distances between the 
delta-function receptor in the CCD (can also be modeled by providing a pinhole filter in front 
of the CCD. In these formula. Ax and Au are the intervals between delta functions in the SLM 
and CCD respectively, f is the focal length, N is the block size and Ax is the placement of the 
delta function in the interval (phase shift) in the SLM. It can be seen that the pixels intervals in 
the CCD and the SLM are not-necessarily the same, which may be implemented by ignoring 
some of the CCD pixels. An alternative matching condition is described below. 

In the example of Fig. 4D, an 8x8 block of image 92 is made symmetric using a 
doubling and mirroring optical system 93 (alternatively to using an SLM) and then 
transformed by a Fourier lens 97. Since the data is mirrored in two dimensions (only one 
shown for clarity) a 8x8 block is transformed into a 16x16 block. The result is then combined 
using a combining optical system 99, to provide an 8x8 DCT fransform. In some embodiments 
of the invention, optical system 93, lens 97 and optical system 99 are combined into a single 
optical element, thus, the end result is a single optical element which performs a DCT, suitable 
for use in lens-matrix 94. A matrix of such optical elements may be combined to form a single 
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optical element which performs a block DCT transform, Altematively to the optical systems 
shown, other constructions can be utilized for mirroring, doubling and combining. In one 
example, an image block is first doubled and arranged as a 2x2 matrix of blocks and then 
individual blocks of the 2x2 matrix are flipped, to provide the symmetry required for the DCT 
transfomi (or other type of transform). 

The correspondence between Fourier transform and DCT can also be utilized for other 
optical transform architectures, for example the matrix-vector multiplication method described 
above. In another example, a wavelet transform can be performed by mirroring data to be anti- 
symmetric instead of symmetric, as in the DCT case. 

The above matching condition may be derived using the following analysis (for a one 
dimensional case). The following equation defines the JPEG-DCT which is to be achieved: 



Assuming symmetric input, where every block of 16 samples is represented as a 
combination of delta functions, spaced at interv^als of size Ax, and transmitted from a 
Ax position inside each interval: 



iV-l=7 
n=0 



/dc{2n + 1) 
2N 



(2) 



yV-l=7 iV-l=7 

2/(;2)-^5'(x-n-Ax- Ax) + X/(n)- J(x + n-Ax-h A-c) 
n=Q n=0 



(3) 



Applying the optical Fourier transform: 



CO 



.2mix 




(4) 



— OO 



The imaginary parts cancel. out (due to the input being symmetric): 




(5) 



Assimiing accurate sampling at the Fourier plane (the CCD): 



n=0 

Since equation (2) is desired, we match: 



s{k)= S /(/2)-cosO 



2;r* A: • Am • (rt * Ax 4- Ax) 



(6) 
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yV--l=7 2/T- ^'•A»•0^•AT + Aa.) _ 

s{k)= 21 /(")cos( ) 

;i=0 

= S /('Ocos( 

«=0 



) 



(7) 



Thus, one matching condition is: 



cos(- ) = cos( ^ (8) 



Leading to: 

Au • Ax 1 



(9) 



and 



Xf 2N 

2-Au-Ax 1 (10) 



Xf 2N 
resulting in the above matching condition: 

'Ax = 0.5 • Ax , J 

In some cases, it may be not be suitable to provide delta functions (pinholes or other 
optical elements) on one or both of the SLM and CCD. The following analysis shows a 
method of matching a CCD and an SLM, by spatially modulating the hght in a less drastic 
manner, for example using continuous neutral density filters. 

The following equation describes an SLM-like object: 

7^^-1=7 ^-1=7 
5(:c)= S/(«)./(x + « Ax) ^^2) 

n=0 «=0 
Where l(x) is a general transmission function of the SLM, assumed identical for all 
pixels, and synnnetric, so it can be mirrored. However, it should be noted that a similar but 
more complex analysis can also be performed in the case where not all the pixels are identical. 
After applying the optical (and continuous) Fourier transform: 
■^-1,=7 2mcn-Ax ^ 
«=0 
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Where L(u) is the Fourier transfoim of l(x). Since the actual sampling is done by summing all 
intensities on a detector cell (i.e., a CCD pixel cell), equation (5) transforms to: 
^ k'Aii-^Au/2 

All 



A;/ ^ 

k'Au-Au/2 (14) 



Where W(u) is the CCD detection weight function. Again, it is assumed that W is the same for 
all pixels but this assumption is not required. Using equation (13): 

k-Au-^Au/2 (N^l=7 ^ ^ ] 

I «=0 ^ J (15) 

Since equation (2) is desired, we match: 

k-Au+Aull 

cos( ' ^ 0 = — J W{u) • L{ti) • cos( ) . du (16) 

2N Au ytf 

k-Au-~Au/2 



We define: 

R(u)= W(u)-L(u)/ Au 
The matching requirement is thus: 



(17) 



k-Au'hAu/2 

cos(— ^^-^)= j i?(iO-cos( — )-du , n,k = 0,l,...N-l 

^ k'Au~Au/2 (18) 

Which results in the following NxN Fredholm I equations (for the ID case. In 2D its 
NxNxNxN equations): 
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k = 0,n = 0: 

Au/2 
1 = Ir{u)c{u 

A/7/2 
^ = 0,7J = 1: 

A»/2 

f 2/T-Ax-z/ 
1= J i?(iOcos( )dic 

Au/2 
k = \,n = 0: 

All + Au/2 
cos(:^)= \Riu)du 
All- Au/2 

k = \,n = V. 

Au + Au/2 
3/r f 2^- Ax ■ u 
cos(— )= J i?(")cos( )du 

Au-Au/2 (19) 



Equation set (19) defines a Fourier coefficients solution to the problem of describing 
R(u) by cosine series, i.e., 

. ^7±(2n + l)^ 2^-u-n-Ax ^ 

R(u)\k= S cos(-^^).cos( ) 

n = 0 ^" 

This solution preferably defines a matching between individual pixels in the SLM (u) 
and the CCD (k). u e[k- Au - Au/2,k • Au + Au/2] . 

It should be noted that equation 20 actually defines a family of solutions, thus, in some 
embodiments of the invention, standard geometries of SLMs and CCDs are used, while in 
others one or both of the SLM and CCD are modified to better fit a particular matching 
solution. In the general case, the matching may be performed by using neutral filters and by 
matching at least the locations, if not the sizes of CCD and SLM pixels. 

In a preferred embodiment of the invention,, the above matching condition(s) are 
applied towards other discrete linear transforms which are to be appUed using Fourier lens: 

Applying the same procedure, as in equations (12)-(20), (18) now reads: 



20 



141/00863 



k-Aii+&.u/2 ^ ^ 

C{k,n)= \ Riti)-cos C^'^. ^ ) du , n.k = 0,l,...N-l 

k-Au-Au/2 (22) 



So for the general ID linear transform: 

R{ii)\, = 2 ") • cosC ^^"",'"'^"^ ), ul^ e [kAu - Au / 2, kAu + Au / 2] 



n=0 



(23) 



or the matching condition of equation (1 1) can be used. In the context of matching conditions 
it should be noted that a matrix of sub-elements is not required. Rather, it is sufficient that 
there be a correspondence between the pixels in the SLM and the pixels in the CCD. A simple 
construction is that of a matrix of elements. 

In an alternative preferred embodiment of the invention, DCT, FFT or block transforais 
are achieved using a holographic lens, for example replacing lens-matrix 94, individual lens- 
elements 95 and/or other optical elements (described below). Alternatively or additionally, two 
dimensional holograms may be used, for example, by providing arrays of phase and amplitude 
modifying materials, instead of refracting elements. Alternatively or additionally, a look-up- 
table based approach to transforming may be used, for example using the look-up table 
methods described in US Patent No. 4,892,370, the disclosure of which is incorporated herein 
by reference. Alternatively or additionally, acousto-optical type optical elements are used. An 
advantage of transform-lens, such as described with reference to Figs. 4A-4D, is that they are 
better matched to the physical model of the compression, i.e., transforming data from an image 
space into a transforai space. Holograms are a general purpose optical element design method, 
which although they are very flexible, may have an efficiency penalty. Look-up tables are 
general purpose solutions which may require a larger and/or more complex optical architecture 
than a matched architecture such as a lenslet array. 

Fig. 5 is a schematic block diagram of an optical JPEG compression system 110, in 
accordance with another preferred embodiment of the invention, in which the DCT 
transformed data is further processed prior to being converted to electrical signals. A main 
difference from the embodiment of Fig. 2 is the provision of an A/D converter 112, which 
converts the data from an analog representation to a digital representation. Thus, coding (e.g., 
VLC and Huffinan) can be performed optically using various types of available hardware 
architectures. The Zig-Zag step (26) may be performed before or after quantization, for 
example, even after the data is converted to electrical signals, by optical sensor 58. An 
exemplary optical A/D converter is described in "Real-Time Parallel Optical Analog-to-Digital 
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Conversion", by A. .^mia^^.A. Sawchuk, T.C. Strand, D. Bo^l and B.H. Soffer, in 
Optics Letters . Vol. 5 No. 3, March 1980, the disclosure of which is incorporated herein by 
reference. 

In the embodiment of Fig. 5, quantization is shown as being performed on the optical 
data, for example utilizing an SLM or a controllable attenuator such as an LCA with one or 
more face polarizers which selectively "divide" DCT coefficients by a weight. Alternatively, 
the data is quantized after the A/D conversion, for example using a suitable lookup table or a 
holographic lens. In embodiments where digital data is represented by spatial bit patterns, as in 
the above paper ("real-time"), quantizing may be performed by spatially blocking out certain 
bits. In embodiments where digital data is represented temporally, temporal filtering may be 
used in which certain pixels are darkened, in synchrony to the bit order of the light pattern, so 
that those bits are blocked out. It is noted that the quantization step and the encoding step (at 
least the VLC) may be combined as a single step, using relatively standard tables, as known in 

the art of electronic JPEG. 

In some embodiments of the invention, it is desirable to achieve different spatial and/or 
bit resolutions for different parts of the image. In one example, the CCD can be read out at 
varying resolutions, responsive to the desired spatial resolution. In another example the light 
source is defocused for portions where a lower resolution is required. Alternatively or 
additionally, the quantization is varied between the blocks. If for example quantization is 
achieved by selective blocking of pixels, this blocking may be implemented using an 
electrically controllable spatial filter, for example an LCD, which can be set to the desired 
quantization. 

In a preferred embodiment of the invention it is desirable to simultaneously generate 
multiple resolutions of JPEG data. In a preferred embodiment of the invention, this is achieved 
by parallel application of the JPEG algorithm, using hardware as described herein. 
Alternatively, this may be achieved (for example in the embodiment of Fig. 2) by reading out 
the CCD at different resolutions, for different JPEG resolution levels. Alternatively, varying 
resolutions may be achieved by zooming the source image 92 up or down, for example using a 
zooming lens or by suitable control of an SLM which generates the Ught. 

Compression of color images may be achieved by converting the image from an RGB 
format into a YUV format (if it is not abready so represented) and then compressing each of the 
Y. U and V components. Typically, only the Y component is compressed at a full spatial 
resolution, while the U and V components are compressed at half their spatial resolution. In 
one preferred embodiment of the invention, different hardware is provided for the different 
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components. Alternatively, the same hardware is used, sequentially. Alternatively, other color 
component separation methods may be used. 

In a preferred embodiment of the invention, an image sequence, such as a video 
sequence is compressed utilizing the above methodology. In a preferred embodiment of the 
invention, each of the images in the sequence sequences is compressed in turn using the above 
method of JPEG compression, providing a series of JPEG compressed images. In a preferred 
embodiment of the invention, inter-frame compression is achieved by motion estimation for 
example using adaptive differential coding by subtracting consecutive images. In a preferred 
embodiment of the invention, consecutive images are subtracted using an SLM which is driven 
with a previous image's density distribution. In a self-electro-optic effect device, the SLM can 
be programmed directly using the previous image, without requiring external electronics to 
store or otherwise manipulate the image. 

Fig. 6 is a simplified schematic flowchart of a base-line method 120 of MPEG- 
compliant compression, which flowchart does not show various feedback loops often utilized 
in the MPEG compression method. One of the main advantages of the MPEG compression 
method over the JPEG compression method is that the MPEG method takes into account 
similarities between consecutive images. One of the main tools for similarity determination is 
motion estimation, in which the motion of portions of the image are determined, so that an 
image can be reconstructed from spatially translated parts of previous images. Transmitting 
only the amount of translation usually requires less bandwidth than transmitting coefficients 
for an entire block. Thus, in an exemplary method, input data is transformed using a DCT 
transform (122). Motion estimation is performed (124). The resulting coefficients of 
translation data is quantized (126), encoded (128) and combined with a header (130) to form a 
data stream. 

In a preferred embodiment of the invention, alternatively or additionally to performing 
the DCT step using optical processing methods, also the motion estimation is performed using 
optical processing. In a preferred embodiment of the invention, motion estimation is 
performed by performing an autocorrelation of the source data with itself, allowing small 
amounts of block motion, using well known optical means, to determine block motion. 
Altematively however, a DCT based motion estimation scheme is used. Thus, a same or 
similar hardware as used for the DCT may also be used for at least part of the motion 
estimation. A method of motion estimation using DCT is described in a Ph.D. Dissertation 
titled "Low Complexity and High Throughput Fully DCT-Based Motion Compensated Video 
Coders", by Ut-Va Koc, presented in 1996 to K.J. Ray Liu of the institute for systems research 
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and sponsored by the Natio^Pscience Foundation Engineering Re^^h Center Program, the 
University of Mar\-land, Harvard University and Industry, in U.V. Koc and KJ.R. Liu, "Low- 
Complexity Motion Estimation Techniques", U.S. Patent No. 5,790,686, issued August 4, 
1998 and in U.V. Koc and K.J.R. Liu, "DCT-Based Motion Estimation", IEEE Trans, on 
Image Processing, Vol. 7, No. 7, pp. 948-965, July, 1998, the disclosures of which are 
incorporated herein by reference. The method described therein can be summarized as follows 
(based on table 4.2 in the Ph.D. dissertation), with the DCT portions preferably being 
performed as described herein. Preferably, other elements of the process are also implemented 
using optical components, for example peak finding. 

a. Compute the 2D DCT coefficients of second kind (2D-DCT-II) of an NxN block of pixels at 
the current frame t, {xt(m,n); m,ne {0, . . . ,N- 1 } } . 

b. Convert stored 2D-DCT-n coefficients of the corresponding NxN block of pixels at the 
previous frame t-1, {xt.i(m,n); m,ne {0,...,N-1}} into 2D DCT coefficients of first kind (2D- 
DCT-I) through a simple rotation unit T. 

c. Find the pseudo phases {gCS(k,l); k=0,l,..,N-l;l=l,2,..,N} and {gSC(k,l); 
k=l,2,..,N;l=0,l,..,N-l}, which are calculated from the DCT coefficients independently at each 
spectral location. 

d. Determine the normalized pseudo phases f(k,l) and g(k,l) from gCS and gSC by setting ill- 
formed pseudo phases to zero. 

e. obtain the inverse DCT (2D-IDCT-II) of f(k,l) and g(k,l) as DCS(m,n) and DSC(m,n) for 
m,ne{0,..,N-l} respectively. 

f. Find peaks in DSC and DCS, which peak positions represent the shift amounts and peak 
signs represent the direction of movement. 

g. Estimate the displacement from the signs and positions of the found peaks. 

It is noted that even in this method of motion estimation, some processing is required 
beyond the DCT, however, a significant portion of the computation may be dealt with by DCT 
or IDCT fransforming of the data (in parallel or in sequence for each block). In a preferred 
embodiment of the invention, the previous image and/or its DCT coefficients are stored and/or 
provided using suitable electronics. Possibly, the optical DCT transforming elements are used 
for performing DCT and IDCT. Alternatively to the above method of motion estimation, direct 
correlation of image blocks may be used to estimate motion, for example, using image 
correlation optical systems known in the art as part of the compression process. 

The above description has centered on compression, however, it should be noted, that 
decompression is very similar to compression and can often utilize similar or the same 
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hardware. In the example of JPEG, DCT (for compression) and inverse DCT (for 
decompression) can be performed using a same optical transform element. In the example of 
MPEG, motion compensation, i.e., recreating images by compensating for the effect of 
motion, which motion was determined using motion estimation, can utilize a similar DCT- 
based method, also described in the above doctorate. It is noted that for some decompression 
methods, there is a requirement for some processing before the transforming of coefficients 
into an image domain. For example, in JPEG de-compression, the compressed image data is 
un-runlength encoded and de-quantized prior to being EDCTed. As with compression, these 
processing steps may be performed optically and/or electronically, depending on the 
implementation. 

Fio. 7 A is a schematic diagram of a direct-compression camera system 200, in 
accordance with a preferred embodiment of the invention. In system 200, an image of a real 
object 202 is acquired directly as transformed and/or compressed data, rather than being 
acquired as image data which is later compressed. There are many applications in which an 
image is acquired in order to be stored on a digital media or in order to be transmitted over 
bandwidth-limited transmission lines. Examples of such applications include digital cameras, 
security cameras, teleconferencing systems, hve-feed TV cameras and video-telephones. In a 
preferred embodiment of the invention, the data is acquired in a compressed marmer, using the 
above described methods of compressing optical data, except that it is preferably the original 
optical waves, arriving from the object, that are compressed, rather than an electronic 
representation which is separately acquired and compressed. 

In the exemplary system of Fig. 7 A, an optional object lens 204 focuses and directs the 
Ught form object 202 onto a DCT lens (or lens matrix) 206. Lens 206 may also comprise other 
optical elements which perform further steps of the image compression method. The processed 
light is collected by a CCD 208 and then further processed and stored in a storage 210. In real- 
time embodiments, the acquired data may be transmitted, instead of or in addition to storage. 
The compression method performed may be a method suitable for still images, such as JPEG 
or a method suitable for moving images. 

In a preferred embodiment of the invention, DCT lens 206 is designed to operate on 
white Ught. Altematively, the light arriving from the scene is filtered so that it is monochrome. 
Alternatively or additionally, the image is acquired under controlled lighting situations, so that 
the light has known characteristics, such as being coherent, monochromatic or formed of a 
small number of narrow spectral bands. Altematively or additionally, the image is acquired 
using a monochromatic light, possible a laser flash, so that the characteristics of the light are 
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controlled by system 200. Wk controlled lighting is especially A for low-light level 
cameras, such as those using GICCD (Gated Intensified CCD) technology. Also, the use of 
coherent lieht simplifies the use of hologram-based image processing techniques. 

Altematively or additionally to an objective lens 204, a light encoding module, such as 
a combination CCD/SLM may be used, to conven incoming light into Ught having desired 
spatial and spectral characteristics. Altematively or additionally, a self-electro-optical effect 
shutter is used, in which the impinging light is used to modulate the transmission of last or 
other controlled light. 

Fig. 7B is a schematic block diagram of a YUV-separated implementation 220 of the 
embodimint of Fig. 7A. Color images may be compressed by separately compressing each 
color component, or more commonly, each of the YUV components. These components may 
be determined using a look-up table or by simple arithmetic on the R, G and B components of 
an incoming image. These separations may be performed using optical means and/or electromc 
means, shown generally as a splitter 222. Each of the resulting color components (224) is then 
processed separately, using a dedicated DCT 206 and a dedicated CCD 208. The results are 
then added using a combiner 226. It should be noted that the U component and the V 
component are usually processed at a lower resolution than the Y component. Thus, the U and 
the V can share optical components. Altematively or additionally, all three components are 
processed using a single optical path, for example on different parts of a same lens-CCD set. 
Alternatively or additionally, the three components are processed sequentially. 

A component which performs image compression or decompression may be packed in 
various ways, depending on the appUcation. In one application, a PC graphics card includes an 
optical processor for aiding in displaying MPEG images. In another example, a CCD camera 
includes an MPEG or a JPEG optical module so that they can provide a compressed data 

5 output as well as a standard data output. 

In a preferred embodiment of the invention, the above described optical elements are 
provided embedded in a transparent substrate, such as a clear plastic or glass, so that once the 
elements are coupled, there is no relative movement due to vibration, heat or other external 
forces. It should be noted that pixel-sized transverse shifts in the optical elements do not 

0 substantially affect the output, providing the SLM can be controlled to shift its image by the 
pixel shift error. In a preferred embodiment of the invention, the optical elements are 
manufactured and tested without a surrounding matrix or with a liquid surrounding matrix, 
which is then soUdified when the relative positions of the optical elements are determined. In a 
preferred embodiment of the invention, the optical processor is calibrated by entering known 
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data and measuring the compressed output. 

Fig. 8 is a schematic diagram of a lithographic implementation of an optical 
compression system 300 in accordance with a preferred embodiment of the invention. An 
advantage of lithographic optics is that they can be fabricated in large batches, and, depending 
on the process, in conjunction with the forming of electronic circuits on a same substrate. It 
should be noted that most lithographic implementations utilize reflective optics rather than 
transmissive optics. The above description has focused on transmissive optics, however, 
reflective optics can also be used for non-lithographic optical processors. Various lithographic 
implementations will occur to a person skilled in the art, however, an exemplary embodiment 
is shown in Fig. 8. 

System 300 comprises generally of a substrate 301, one or more reflective surfaces 303 
which are etched and/or otherwise lithographically processed or micro-machined to form 
reflective optical elements and an interposing clear medium 305. An SLM or a diode array 302 
is used to provide an image. The light is reflected off substrate 301 to a reflective DCT lens 
304. The transformed light is reflected back to substrate 301 and then to a CCD or other 
optical array detector 306. Optionally, the CCD array or other optical, electrical or electro- 
optical elements may be formed directly on the substrate, for example as indicated by 
reference 308. In one example, a quantizer, or a holographic reflecting lens are formed at 
location 308. Possibly, reference 308 indicates an active element, such as an LCD array. 

In a preferred embodiment of the invention, device 300 is manufactured to DCT a 
single 8x8 block rather than a whole image. A plurality of systems 300 is preferably used to 
compress an entire image. Altematively, system 300 is manufactured to process a single vector 
rather than an array. Although system 300 may form a part of a dedicated JPEG or MPEG 
decoder or encoder, in a preferred embodiment of the invention, one or more system 300 type 
elements are used for the construction of digital signal processor or other integrated circuits, 
for example to assist in high-end graphical applications. 

The above description has centered on DCT based compression methods. However, 
other transform based compression methods may also be implemented in accordance with 
pj-ef^^eci embodiments of the invention. In one example, a wavelet compression method is 
implemented using a block DWT (discrete wavelet transform). Possibly, there is an overiap 
between blocks. Such a transform is described, for example in G. Strang and T. Nguyen, 
"Wavelets and Filter Banks", Wellesly-Cambridge Press, 1997, pp. 502, the disclosure of 
which is incorporated herein by reference. Preferably, such a wavelet compression 
implementation includes bit-plane coding techniques such as SPIHT or EZW, possibly 
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If 

implemented usmg a lookuplable. 

The above description has centered on image compression, however, in accordance 
with a preferred embodiment of the invention, optical components are used for compressing 
other types of signals, for example, audio signals. It is noted, however, that image compression 
is generally more suitable for transform based compression and being t^vo-dimensional, is 
more computationally-complex to compress than other types of data. 

In the above detailed description, various types of optical data representations are 
suggested, as well as various types of optical systems. In a preferred embodiment of the 
invention, the optical representation used is selected to match the optical system, for example, 
an analog representation for an analog system, in some cases, the data may be converted 
between representations, to take advantage of particular optical configurations, for example 
digital optical data may be converted into analog optical data to use a particular lenslet-based 
implementation of a DCT transforming element. 

The optical processing hardware is preferably dedicated for particular tasks. 
Alternatively, in some preferred embodiments of the invention, the same hardware 
components may be used for different steps in a process (such as a DCT component for 
compression and for motion estimation), for different processes (such as compression and 
decompression) and/or for different data blocks in a same process, such as in serial processing 
of data blocks). Alternatively or additionally, the hardware may be programmable, at least to 
some extent. For example, by modifying the behavior of an SLM and a CCD which form part 
of a Fourier-based data transform optical component, different type of transforms can be 
achieved with a single hardware, for example, DCT and DST. Alternatively or additionally, 
the matching layer may be programmable, for example being an addressable LCD, so that the 
size and/or location of pinholes can be controlled. Alternatively or additionally, by controlling 
the opacity of single LCD cells different continuous spatial filtering configurations can be 
achieved. 

It will be appreciated that the above described methods of optical processing may be 
varied in many ways, including, changing the order of steps, which steps are performed using 
electrical components and which steps are performed using optical components, the 
representation of the data and/or the hardware design. In addition, various distributed and/or 
centralized hardware configurations may be used to hnplement the above invention. In 
addition, a multiplicity of various features, both of methods and of devices, have been 
described. It should be appreciated that different features may be combined in different ways. 
In particular, not all the features shown above in a particular embodiment are necessary in 
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every similar preferred embodiment of the invention. Further, combinations of the above 
features are also considered to be within the scope of some preferred embodiments of the 
invention. In addition, the scope of the invention inckides methods of using, constructing, 
caUbrating and/or maintaining the apparatus described herein. When used in the following 
claims, the temis "comprises", "comprising", "includes", "including" or the like mean 
"including but not limited to". 
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CLAIMS 



1 . A method of image compression, comprising: 
providing image-data encoding light; 

transforming said light from an image space to a transform space utiHzing an optical 
component; and 

converting said transformed light into electrical signals, which electrical signals 
represent a compressed representation of said image data. 

2. A method according to claim 1, wherein said converting comprises converting after 
quantization. 

3 . A method of image de-compression, comprising: 
providing compressed-image-data encoding light; 

transforming said light from a transform space to an image space utilizing an optical 
component; and 

converting said transformed light into electrical signals, which electrical signals 
represent said image data. 

4. A method according to any of claims 1-3, wherein said transforming comprises 
transforming using one or more optical elements which perform a block transform. 

5. A method according to claim 4, wherein said one or more optical elements comprises a 
matrix of optical lens elements. 

6. A method according to claim 5, wherein at least one of said optical lens elements 
comprises a lenslet array which performs a block transform for a single block. 

7. A method according to claim 4, wherein said one or more optical elements comprises 
an element which performs a vector by matrix multiplication. 

8. A method according to claim 4, wherein said one or more optical elements comprises 
an element which generates a matrix product. 



30 



141/00863 




9. 



A method according to claim 8, wherein said matrix product comprises a triple matrix 



product. 



10. A method according to claim 4, wherein said transforming utilizes at least one 
electrical component for transferring data between said at least one optical component. 

11. A method according to any of claims 1-10, wherein said image is compressed using a 
JPEG standard. 

12. A method according to any of claims 1-10, wherein said image is compressed using an 
MPEG standard. 

13. A method according to any of claims 1-10, wherein said image is compressed using a 
wavelet compression method. 

14. A method according to any of claims 1-13, wherein said image data represents an 
image sequence. 

15. Apparatus for direct acquisition of compressed image data, comprising: 

at least one optical element which receives light waves from an imaged object and 
which transforms said image into a transform space; and 

an optical detector which receives said transformed light and generates electronic 
signals responsive to said received light. 

16. Apparatus according to claim 15, comprising an optical quantizer which quantizes said 
transformed light prior to its reception by said detector. 

17. Apparatus according to claim 15, comprising an electronic quantizer which quantizes 
said electronic signals to produce compressed data. 

18. Apparatus according to claim 15, wherein said optical detector quantizes said light. 

19. Apparatus according to any of claims 15-18, wherein said at least one optical element 
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comprises a single optical e^fcnt which performs said transforming 



20. Apparatus according to any of claims 15-19, comprising a storage unit for storing at 
least one set of said electronic signals, corresponding to a still image. 

21. Apparatus according to any of claims 15-19, comprising a storage unit for storing at 
least one set of said electronic signals, corresponding to an image sequence. 

22. Apparatus according to any of claims 15-21, wherein said transform space comprises a 
DCT transform space. 

23. Apparatus according to any of claims 15-21, wherein said transform space comprises a 
wavelet transform space. 

24. A block-transfomi optical element, comprising: 

a plurality of optical groups each group comprising at least one optical sub-element, 
wherein each of said optical groups transforms a block of data, 

25. An element according to claim 24, wherein said at least one sub-element comprises a 
lenslet array portion. 

26. An element according to claim 24 or claim 25, wherein said plurality of optical groups 
are arranged as a matrix. 

27. An element according to any of claims 24-26, wherein said element performs a JPEG- 
DCT transform. 

28. An element according to any of claims 24-26, wherein said element performs an 
inverse DCT transform. 

29. An element according to any of claims 24-26, wherein said element performs a wavelet 
transform. 

30. An element according to any of claims 24-29, wherein said optical element performs 
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said transform on dataarranged as a spatially impulse image. 

31. An element according to any of claims 24-27, wherein said optical element performs 
said transform on data arranged as a continuous image. 

32. An element according to any of claims 24-31, wherein said transform comprises a 
linear transform. 

33. An element according to any of claims 24-32, wherein said transform comprises a non- 
separable transform. 

34. An element according to any of claims 24-33, wherein said optical groups overiap. 

35. An element according to any of claims 24-33, wherein said optical groups no not 
overlap. 

36. A method of DCT-like transforming of data, comprising: 
providing optical waves which encode said data; and 

passing said waves through at least one optical element, which optical element 
modifies said optical waves into second optical waves carrying a transform of said data. 

37. A method according to claim 36, wherein said transform comprises a block transform. 

38. A method according to any of claims 36 or 37, wherein said at least one optical 
element is an atomic optical element. 

39. A method according to any of claims 36 or 37, wherein said at least one optical 
element comprises a pluraUty of optical elements. 

40. A method according to any of claims 36-39, wherein said at least one optical element 
comprises a discrete optical processing element. 

41. A method according to any of claims 36-39, wherein said at least one optical element 
comprises substantially only continuous optical processing element. 
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42. A method according to any of claims 36-41, wherein said transform is a JPEG-DCT 
transform. 

43 . A method according to any of claims 36-41, wherein said transform is an inverse DCT 
transform. 

44. A method of optical motion estimation, comprising: 

providing a first set of light waves encoding data of a first image block; 
DCT transforming said first set of light waves using an optical component; 
providing a second set of light waves encoding data of a second image block; 
DCT transforming said second set of light waves using an optical component; and 
processing said first and said second DCT transformed data sets to determine an 
amount of relative motion of said two blocks. 

45. A method according to claim 44, wherein both of said DCT transforms utilize a same 
optical component for said transforming. 

46. A method according to claim 44 or claim 45, wherein said processing comprises 
electronically processing said transformed data sets. 

47. A method according to any of claims 44-46, comprising utilizing said determined 
motion for generating a compressed image stream. 

48. A method of transforming data using a discrete transform, comprising: 
preprocessing said data to yield optically encoded data; 

Fourier transforming said data using a Fourier lens; and 
postprocessing said data to yield transformed data. 

49. A method according to claim 48, wherein said preprocessing comprises optically 
preprocessing. 

50. A method according to claim 49, wherein said preprocessing comprises matching said 
data to said postprocessing. 
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51. A method according to claim 50, wherein said postprocessing comprises acquiring said 
data and wherein said matching comprises matching to said acquisition. 

52. A method according to claim 51, wherein said postprocessing comprises spatially 
modulating said hght, 

53. A method according to claim 48, wherein said preprocessing comprises spatially 
modulating said light. 

54. A method according to claim 52 or claim 53, wherein said spatially modulating 
comprises masking said data using an array of pinholes, 

55. A method according to claim 52 or claim 53, wherein said spatially modulating 
comprises continuously varying said amplitude of said data. 

56. A method according to any of claims 48-55, wherein said preprocessing comprises 
mirroring said data. 

57. A method according to any of claims 48-56, wherein said transform comprises a JPEG- 
DCT transform. 

58. A method according to any of claims 48-56, wherein said transform comprises a 
wavelet transform. 

59. A method according to any of claims 48-56, wherein said transform comprises an 
inverse DCT transform. 

60. Apparatus for compressed image data manipulation, comprising: 
an input for receiving data; 

at least one optical component for transforming said data between an image space and a 
transforai space; and 

an output for outputting said manipulated data. 
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61. Apparatus accordin^^claim 60, wherein said received datJfcnprises uncompressed 
image data and comprising a processor for processing said data after said transform to yield 
compressed image data to be outputted. 

62. Apparatus according to claim 60, wherein said received data comprises compressed 
image data and comprising a processor for processing said data before said transform to yield 
transform data to be transformed by said optical component for said output. 

63. Apparatus according to claim 62, wherein said processor includes an optical D/A 
converter. 

64. Apparatus according to claim 62, wherein said processor comprises an optical 
multiplier for de-quantizing data. 

65. Apparatus according to claim 61 or claim 62, wherein said processor performs at least 
some of its processing on optic waves. 

66. Apparatus according to any of claims 61-65, wherein said processor performs at least 
some of its processing on electronic signals. 

67. Apparatus according to any of claims 60-66, comprising an optical motion estimator. 

68. Apparatus according to claim 67, wherein said optical motion estimator utilizes said 
optical component for optical motion estimation. 

69. Apparatus according to claim 67 or claim 68, wherein said optical motion estimator 
includes at least one electronic component. 

70. Apparatus according to any of claims 60-68, comprising an optical motion 
compensator. 

7 1 . Apparatus according to claim 70, wherein said optical motion compensator utilizes said 
optical component for optical motion compensation. 
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72. Apparatus according to claim 70 or claim 71, wherein said optical motion compensator 
includes at least one electronic component. 

73. Apparatus according to any of claims 60-72, wherein said data is compressed using a 
JPEG compression. 

74. Apparatus according to any of claims 60-72, wherein said data is compressed using a 
wavelet compression. 

75. Apparatus according to any of claims 60-72, wherein said data is compressed using an 
MPEG compression. 

76. Apparatus according to any of claims 60-75, wherein said data comprises an image 
sequence. 

77. Apparatus according to any of claims 60-74, wherein said data comprises is a single 
image. 

78. Apparatus according top any of claims 60-77, wherein said data is manipulated as 
blocks. 

79. Apparatus according to claims 78, wherein at least two of said blocks are manipulated 
in parallel. 

80. Apparatus according to claims 78, wherein at least two of said blocks are manipulated 
in series and comprising a buffer circuit for directing said blocks in series into a single optical 
component. 

81. Apparatus according to any of claims 60-80, wherein said optical component 
comprises a discrete data transforming optical element. 

82. Apparatus according to any of claims 60-81, wherein said optical component 
comprises a holographic lens. 
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53. Apparatus accordin^^ any of claims 
comprises an acousto-optic element. 

54. Apparatus according to any of claims 
comprises an optical analog to digital converter. 

85. Apparatus according to any of claims 
comprises an optical digital to analog converter. 

86. Apparatus according to any of claims 
comprises matrix product calculating element. 



60-82, wherein 




optical component 



60-83, wherein said optical component 



60-84, wherein said optical component 



60-85, wherein said optical component 



For the applicant, 
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Fenster & Co. Patent attorneys Ltd. 
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